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Abstract 


The aim of research was validation of a method for the assessment of cognitive complexity of 
chemical technology problem tasks. The method included an assessment of the difficulty of concepts 
and an assessment of their interactivity. As a research instrument for assessing performance, 
test of knowledge was used. Each task in the test was followed by a 5-point Likert scale for the 
evaluation of invested mental effort. The validity of this method was confirmed by a series of linear 
regression analysis where extremely high values of correlation coefficients are obtained among 
the examined variables: student's performance, invested mental effort and cognitive complexity. 
Keywords: cognitive complexity, problem solving, chemical technology. 


Introduction 


Although the problem of unpopularity of chemistry often lies in old teaching 
methods with the absence of laboratory exercises in primary and secondary schools 
(Woldeamanuel, Atagana & Engida, 2014), understanding of chemical concepts at three 
levels of representation (Chiu, Chou, & Liu, 2002), one of the most important causes 
of this phenomenon is that the students of chemistry do not have direct contact with 
its application in a real environment or industry (Lundgren, 2006). This is especially 
noticeable among students of chemistry and related disciplines. They come into contact 
with a larger number of laboratory work at university. Many of them have the problem 
because theoretical knowledge has already been adopted and it is difficult to integrate 
theoretical knowledge with the practical application of the learned chemical processes. 
After the industrial revolution, which is considered the beginning of the development 
of chemical technology, it became necessary to transform simple, applicable practical 
(crafted) knowledge into science-based learning, especially for the high technological 
education needs (Lundgren, 2006). Due to the contribution that chemical technology 
has in the production of useful materials and the development of efficient technologies 
that serve both the individual and society as a whole, the utmost importance is to 
implement knowledge of chemical concepts and processes through education (Hofstein 
& Kesner, 2006). Chemical technology is in touch with many scientific disciplines such 
as economics, physics, mathematics, cybernetics, applied mechanics, environmental 
protection and other technical sciences. Because of this kind of the correlation, chemical 
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technology possesses a special challenge in terms of dimensioning the complexity of its 
problem tasks, since they may contain several different concepts from different subjects. 

As a reliable method for calculation of the rating of cognitive complexity of 
tasks, mostly because the subjectivity of experts is reduced to the minimum, is a method 
developed by Knaus, Murphy, Blecking and Holme (2011). The Rubrics for the rating 
of the cognitive complexity of the problems, such as this specifically designed for this 
method, need to be created wherever possible for different fields of chemistry as well as 
for different levels of education. In other words, it is necessary to implement as many 
of its adaptations as possible with detailed and additional analysis of basic concepts 
and additional concepts that are specific to a specific subject. Various Rubrics have been 
developed for assessment of the cognitive complexity of chemistry problems (Knaus et 
al., 2011; Raker, Trate, Holme, & Murphy, 2013; Horvat, Segedinac, Milenkovic, & Hrin, 
2016; Horvat, Rodic¢, Segedinac, & Ronéevic, 2017). Cognitive complexity calculated 
using Rubrics in all cases is highly correlated with students’ achievement and students' 
invested mental effort. Reserach aim of this paper was to create and validate a method for 
the assessment of cognitive complexity of chemical technology problem tasks. 


Research Methodology 
General Background 


As the specificity of the chemical technology problems largely inlude mass 
balance with and without chemical reaction, as well as energy balance, these domains 
were the basic concepts represented in the Table for assessment of concept difficulty. 
By estimating the difficulty of the concepts represented in the problem (easy, medium 
and hard), the Table for estimating the difficulty of concepts and their interactivity was 
created for chemical technology problems, using the method from Knaus et al.(2011) 
where the numerical rating of the cognitive complexity of the problem was calculated. 


Sample Selection 


The total sample of this research consisted of two classes of chemistry students 
at the Department of Chemistry, Biochemistry and Environmental Protection, Faculty of 
Sciences, Novi Sad, Serbia. Students of these classes according to the curriculum of the 
Faculty of Science study the subject of Chemical Technology at the second or third year 
of their studies (basic academic studies in these classes last 4 years). 


Instrument and Procedures 


The test with five tasks was designed as a research instrument for the purpose of 
this research. Each correct task was evaluated with one point, so the maximum score 
on the test was 5 points. The numerical rating of the cognitive complexity of the tasks 
was in range from | - 5. Besides the achievement, assessment of the invested students’ 
mental effort for each student was measured. For these purposes, a 5-point Likert scale 


was used. 
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Data Analysis 


The obtained data were analyzed using Statgraphics Centurion XVI and IBM 
SPSS Statistics 22 software programs. 


Research Results 


The test showed a good reliability which was calculated as a measure of internal 
consistency and expressed as a Cronbach o coefficient. The test was moderate difficulty 
level (the average achievement is 1.86 / 5) and the excellent index of discrimination 
(0.62 for achievements and 0.70 for mental effort). The average value of the mental effort 
on the test was 3.32 which means that the test is not difficult or easy on the 5-point Likert 
scale. 

Information about the validity of Rubrics for the cognitive complexity rating 
was obtained by combining the measures of students’ achievements and the measure 
of the investeded mental effort. In order to validate this procedure for the assesment 
of cognitive complexity of chemical technology problems, the existence of statistically 
significant correlations between students' achievement and cognitive complexity of the 
problem, students' invested mental effort and cognitive complexity, as well as student 
achievement and invested mental effort. High correlation coefficients were obtained: 
-.61 for dependence achievement - mental effort; -.46 for dependence achievement - 
cognitive complexity and .58 for dependence mental effort- cognitive complexity). The 
procedure was validated with statistically significant correlations. 


Conclusions and Implications 


With optimization of the cognitive complexity of the tasks, which means specially 
designing the Rubric for the assessment of the cognitive complexity of tasks in chemical 
technology, the requirements for information processing are reduced and in this way it 
affects on better students'achievement with a minimal mental effort. Based on the obtained 
correlation coefficients, there is shown that with increasing of cognitive complexity of 
the problem, the students use more resources of working memory, and therefore invest a 
higher mental effort to solve the task and have lower achievement that is in accordance 
with the results previously obtained. As a possibility of application can be distinguished 
to create other Rubrics in different domains, as well as validation of these Rubrics by 
another method. Also, a wide range of complex problems can be created in chemical 
technology, which requires skills and concepts needed to solve them, such as a large 
number of concepts of thermodynamics, the process streams and others. 
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